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1 SUMMARY 

This paper describes the capabilities and evolution of 
a flight-test engineer’s workstation (called TEST .PLAN) 
from an automated flight-test management system. The 
concept and capabilities of the automated flight-test man- 
agement system are explored and discussed to illustrate 
the value of advanced system prototyping and evolution- 
ary software development. 

2 NOMENCLATURE 

ART automated reasoning tool 

ATMS automated flight-test management system 

dof degree of freedom 

FTE flight-test engineer 

FTTC flight-test trajectory controller 

FTTG flight-test trajectory guidance 

GUI graphical user interface 

HARV High Alpha Research Vehicle 

RDBMS relational database management system 

TACT tactical aircraft technology 

3 INTRODUCTION 

This paper describes the development and capabilities of a 
flight-test engineer’s workstation called TEST .PLAN and 
its evolution from the automated flight-test management 
system (ATMS). The ATMS was a tool for flight-test 
planning and scheduling; it contained expert systems for 
maneuver ordering, range management, and maneuver re- 
quirements evaluation. These expert systems were com- 
bined with three and six-degree-of-ffeedom simulations, 
state-of-the-art trajectory optimization, and a powerful 
graphic user interface to provide a desk top workstation. 

•PRC Inc., Edwards, California 
**GAC Systems Inc., San Juan Capistrano, California 


TEST-PLAN is a computer program designed to run on 
standard graphics workstations as an aid to flight-test en- 
gineers (FTEs) in planning and executing flight-test pro- 
grams. TEST-PLAN allows the FTE to organize and file 
extensive amounts of planning data while satisfying plan- 
ning requirements on a flight-by-flight basis using air- 
craft and flight-specific information about instrumentation, 
telemetry, range, center-of-gravity, airborne and ground 
support, aerodynamic configuration, system configuration, 
and payload. 

TEST.PLAN is the result of several generations of evo- 
lution. Originally combined with a maneuver autopilot, 
the first version of the ATMS was designed for flight- 
test maneuver planning and scheduling as well as ma- 
neuver execution and real-time flight-test monitoring; this 
first version of the ATMS was demonstrated in October 
1987 using the NASA simulation facility at the Dryden 
Flight Research Facility. A second workstation version of 
ATMS evolved from lessons learned from the preliminary 
version — this second version eliminated the maneuver au- 
topilot concept but retained a real-time flight monitoring 
capability; version two of the ATMS was demonstrated in 
mid- 1990 at NASA using the F-18 High Alpha Research 
Vehicle (HARV) flight-test plans. A third commercial ver- 
sion of ATMS (called TEST-PLAN) resulted from the ear- 
lier experience and is designed as a FTE aid in planning 
and executing flight-test programs; TEST .PLAN is cur- 
rently being used or considered for use by United States 
and international flight-test organizations. 

4 THE AUTOMATED FLIGHT-TEST MANAGE- 
MENT SYSTEM 

The ATMS was originally developed at the NASA Dry- 
den Right Research Facility as a part of the NASA Air- 
craft Automation Program-a program focused on apply- 
ing interdisciplinary state-of-the-art technology in artificial 
intelligence, control theory, and systems methodology to 
problems of operating and flight testing high-performance 
aircraft In this section we present the background and a 
description of the ATMS [1,2,3]. 
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4*1 Background of the Automated Flight-Test 
Management System 

The ATMS was an outgrowth of the flight-test trajectory 
guidance (FTTG) work performed over the past decade on 
such programs as the F-lll Tactical Aircraft Technology 
(TACT) Program, the F-15 Propulsion/Airframe Integra- 
tion Program, and the F-15 10°-Cone Program [4]. The 
FTTG provided display information to the pilot to allow 
complex, demanding flight research maneuvers to be flown 
more accurately. The FTTG was extended to a closed- 
loop system for the Highly Maneuverable Aircraft Tech- 
nology (HiMAT) Program flight-test maneuver autopilot 
(FTMAP) [5]. In conjunction with this flight research at 
Dryden, Integrated Systems, Inc., under contract to NASA 
has developed a design methodology for these types of 
controllers [6,7,8] which has resulted in the basis of a 
flight-test trajectory controller (FTTC) which was flight 
tested in early 1990 on the F-15 Highly Integrated Digital 
Electronic Control (HIDEC) aircraft [9]. This FTTC was 
a major component of the ATMS as originally conceived 
and implemented. 

The ATMS project was structured around a flight-test 
scenario and was an extension of work performed by 
SPARTA, Inc., (SPARTA, Inc., Laguna Hills, CA) un- 
der contract to NASA defining the need for a Na- 
tional Remote Computational Flight Research Facility 
(NRCFRF). The work on the NRCFRF contract defined 
the need for an expanded remotely augmented vehi- 
cle (RAV) capability and a flight program to demon- 
strate that capability. In the ATMS, a range, en- 
ergy, and flight-test monitor expert system was used 
in conjunction with the FTTC to order maneuvers 
by priorities and energy management considerations 


while restricting the vehicle to the confines of a specified 
Edwards AFB test range. This expert system could be used 
online to control the research aircraft in flight and monitor 
the progress of a flight test; or offline as a planning tool for 
ordering the test maneuvers for a flight. The expert system 
used predictions of maneuvers based on simulation models 
for planning and actual flight-test data measurements for 
real-time vehicle control, data monitoring and flight test 
management 

4 J2 Components of the Automated Flight-Test 
Management System 

The main components of the ATMS were a trajectory con- 
troller based on the FTTC system [7,8], a flight-test plan- 
ning expert system, a man-machine interface, and a flight- 
test monitoring expert system. The partitioning of func- 
tions in the ATMS was designed with two goals in mind; 
minimizing the bandwidth of the communication between 
components, and appropriate distribution of functions be- 
tween numeric and symbolic processing. 

The components described in this section perform the flight 
planning and monitoring functions. The fully developed 
ATMS (Fig. 1) was expected to perform program plan- 
ning, block planning, and in-flight replanning which are 
not described herein as they were never implemented in 
the first ATMS. 

4 2.1 Trajectory Controller 

The trajectory controller was a collection of outer-loop 
guidance control laws which provide precise control for a 



Fig. 1 ATMS functions. 








vehicle performing high-quality flight research maneuvers 
such as level accelerations, wind-up turns, and pushover- 
pullup maneuvers. The trajectory controller was algorith- 
mic, implemented in FORTRAN 77, and executed on a 
numeric processor. 

The interface between the trajectory controller and the re- 
maining components of the ATMS was designed to min- 
imize the bandwidth of the communications across that 
interface. The trajectory controller accepted input com- 
mands consisting of an ordered list of maneuvers by type. 
Each maneuver consisted of a trim point, maneuver con- 
ditions, and end conditions. These commands contained 
from three to seven parameters each. 

Once maneuver commands were received by the trajec- 
tory controller, the controller operated independently of the 
ATMS until another command list was received. The tra- 
jectory controller generated trajectories and trajectory fol- 
lowing controls based on the maneuver commands and the 
aircraft instrumentation. 

4^2 Flight-Test Planning Expert System 

Flight-test planning must be done at several levels. At the 
highest level, the flights required for an entire program are 
established by the project requirements. At the next level, 
blocks of flights are determined by a more detailed analy- 
sis of the project requirements and are partitioned accord- 
ing to similarity of prerequisites, flight envelope require- 
ments, and test needs to establish an orderly progression of 
blocks of flights satisfying the high-level project require- 
ments. Within each block a number of individual flights are 
identified based on the detailed analysis of maneuvers re- 
quired to satisfy the block requirements. Individual flights 
are then identified with a number of these maneuvers and 
the FTE must order maneuvers within a flight based on 
considerations of range, fuel, and energy management, as 
well as maneuver priorities. 

The ATMS implemented only the test planner expert sys- 
tem. The test planner accepted a list of maneuvers and or- 
dered them using rules that considered maneuver priorities, 
energy management, test range boundaries, and envelope 
limitations Maneuvers which could not be included in the 
flight plan were eliminated from the plan being developed. 

The flight-test planning expert system accepted test plan 
inputs from the FTE using a menu driven and icon based 
man-machine interface or previously stored test plan en- 
tries. When the list of test maneuvers was entered into 
the ATMS, the FTE selected the flight-test planning expert 
system which then used its knowledge base to order ma- 
neuvers, prioritize maneuvers, and construct a trajectory. 
As each maneuver was added to the planned trajectory, it 
was tested to insure that no system constraints had been 
violated. When constraint violations occurred, the flight- 
test p lanni ng expert system displayed information to the 
FTE describing the constraint violations and provided an 
explanation of the constraint, if requested. Maneuver pri- 
ority was extremely important when fuel constraints were 
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tested; lower priority maneuvers were removed from the 
test plan to satisfy fuel constraints. 

The flight-test planning expert system was developed using 
the Automated Reasoning Tool (ART) expert system de- 
velopment environment hosted on a symbolic computer 
with a numeric processor board. It contained over 
200 rules. 

4 23 Man-Machine Interface 

The man— machine interface component of the ATMS pro- 
vided a means of information entry and display. This in- 
terface was used during flight planning and flight plan exe- 
cution. The main display had three major components: the 
map. timeline, and command menu. In the map section of 
the main display were two types of displays: the trajectory 
planning display (Fig. 2), and the trajectory map display 
(Fig. 3). These map displays presented a two-dimensional 
view of the test range with the aircraft trajectory superim- 
posed. The stored map was larger than the portion pre- 
sented on the display. Pan and scroll were accomplished 
by using the mouse to choose an appropriate button de- 
picted across the top of the display. A“navigate” button 
was also included to quickly determine course and distance 
between present aircraft position and any point within the 
stored map. The timeline component of the main display 
presented information on the aircraft trajectory in terms of 
altitude as a function of time or events. Figure 4 shows a 
timeline display of altitude as a function of time. Timeline 
scroll buttons allowed the FIE to examine different time or 
event segments by scrolling the timeline. The command 
menu portion of the main display allowed the user to se- 
lect (using “mouse” or keyboard inputs) ATMS operational 
modes, maneuvers, or explanations of ATMS actions. 

The man-machine interface was rule based with over 
200 rules and presented on the computer monitor and key- 
board. The interface was developed in ART. 

4.2.4 Flight-Test Monitor Expert System 

The flight-test monitor expert system provided an inter- 
face between the FTE and either the planned trajectory 
or the actual trajectory (whether generated by simulation 
or flight). This system also provided the trajectory con- 
troller with inputs from the list of maneuvers in the planned 
trajectory. 

The flight-test monitor expert system issued maneuver 
requests to Ihe trajectory controller, then monitored the air- 
craft parameters of interest to insure that no system con- 
straints were violated. This system also monitored ma- 
neuver quality. When a system constraint was violated or 
the quality of a maneuver was unacceptable, the flight-test 
monitor expert system notified the FTE of the problem and 
made recommendations based on the information within 
its knowledgebase. Each maneuver was selected from the 
list of planned maneuvers in order; the flight-test monitor 
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Fig. 3 Trajectory map display. 


expert system initiated these maneuvers and then waited 
for the trajectory controller to finish a maneuver before 
proceeding to the next maneuver on the list. 

4J Automated Flight-Test Management System 
Configurations 


The ATMS had three configurations: the FTE workstation, 
the simulation validation system, and the Sight system. 
The FTE workstation and the simulation validation system 


were used to develop and evaluate flight-test plans. The 
simulation validation system was also used to aid in the 
validation of the flight system including aircraft modifica- 
tions. The flight system was used to actually conduct flight 
test by executing the flight-test plan, monitoring the perfor- 
mance of the aircraft, and controlling the aircraft in flight. 


4.3.1 Flight-Test Engineer’s Workstation 


The configurations of the FTE workstation is shown in 
Figure 5. This system was used by the FTE to develop 
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Fig. 4 Timeline display. 


preliminary flight-test plans without having to use the air- 
craft simulator. This provided the FIB with a stand- 
alone system that was separated from the aircraft simulator, 
which was always in great demand, and thus allowed more 
flexibility in test plan development. 

The FIB workstation included two computers; a symbolic 
computer with a numeric processor board and a graphics 
workstation. The LISP processor on the computer con- 
tained the flight-test planning expat system, the man- 
machine interface system, and the rule-based portion of the 
flight-test monitoring expert system. The numeric proces- 
sor on the symbolic computer contained a three degree -of- 
freedom (3 dof) digital performance simulation (DPS) and 
the software to execute the algorithmic, trajectory manage- 
ment portion of the flight-test management expert system. 
The LISP processor and numeric processor board commu- 
nicated using an internal bus. The graphics workstation 
contained a six degree -of-freedom (6 dof) simulation of the 
aircraft and the FTTC. The two computers communicated 
using Ethernet with a standard protocol. 

4.3.2 Simulation Validation System 

The configuration of the simulation validation system is 
shown in Figure 6. This system was used by the FTE to 
evaluate flight plans developed on the FTE workstation 
to provide detailed pilot-in-the-loop mission briefing and 
familiarization, and as a validation facility for testing the 
ATMS as well as the ground and aircraft systems to be used 
in the actual flight testing. 

The simulation validation configuration of the ATMS in- 
cluded three computers; the symbolic computer and two 
real-time mini computers. The computer in the simula- 
tion validation system was configured identically to the 


FTE workstation configuration of this processor. One mini 
compntw (designated the “control law computer”) con- 
tained the trajectory controller software and communicated 
with the symbolic computer using a standard protocol. The 
communication between this mini computer and the sym- 
bolic computer was identical to the communication be- 
tween the computer and the graphics workstation in the 
FTE workstation configuration. The other mini computer 
contained a detailed 6 dof simulation of the aircraft and 
also contained detailed models of the downlink and uplink 
telemetry system. The two mini computers communicated 
in engineering units through FORTRAN named common 
blocks using a two-port shared memory. 

43.3 Flight System 

The configuration of the ATMS flight system is shown in 
Figure 7. The flight system was to be used to conduct flight 
test by executing the flight test plan, monitoring the perfor- 
mance of the aircraft, and controlling the aircraft in flight. 

The flight system configuration of the ATMS included 
three computers; a symbolic computer and two mini com- 
puters. The computer in the flight system was configured 
identically to the FTE workstation and simulation valida- 
tion system configurations of this computer. One mini con- 
trol law computer contained the trajectory controller soft- 
ware and communicated with the computer using a stan- 
dard protocol. The communication between the control 
law computer and the smaller computer was identical to 
the communication between the smaller computer and the 
control law computer in the simulation validation system 
configuration. A second mini computer (designated the 
“engineering units computer”) was included in the flight 
system and provided processing required for the uplink and 
downlink telemetry systems. The communication between 
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Fig. 5 FTE workstation configuration. 


the two mini computers was identical to the communica- 
tion between the two mini computers in the simulation val- 
idation configuration. In the flight system, the simulation 
model of the aircraft and telemetery systems were to be 
replaced with actual systems. 

5 THE EVOLUTION OF TEST-PLAN FROM THE 
AUTOMATED FLIGHT-TEST MANAGEMENT 
SYSTEM 

The first version of the ATMS was used to develop the 
rapid prototyping facility for flight research in advanced 


systems concepts [2,10]. This rapid prototyping facility 
was intended to allow easy t ransit ion f rom concept to sim- 
ulation then to flight Not only was ATMS the first system 
to be used in this facility, it was the first syste m to bene fit 
from the capabilities provided by this facility. .. - T 

As originally conceived, ATMS was to have combined sev- 
eral concepts into a single system that would allow plan- 
ning, simulation, execution, and monitoring of research 
test flights. But this was una chievable f or several reasons. 
The most apparent problem was the inadequacies of the 
symbolic processors and the expat system development 
language when applied to real-time tasks. Without this 
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Fig. 6 Simulation validation system configuration. 


facili ty these problems might not have been detected until 
much later in the development program. 

The problem of computers and expat system development 
languages was addressed by re-implementing the expert 
systems using CLIPS (*C Language Production System), 
converting from LISP to ‘C,’ using X-windows for the 
graphical user interface, and re-hosting the system on stan- 
dard numeric workstations. 

Finally, experience in the rapid prototyping showed the dif- 
ficulties inherent in a system as ambititious as ATMS. In- 
stead of a single system to manage all aspects of planning. 


simulation, execution, and monitoring, we decided to de- 
velop several separate but compatible systems. Thus, the 
rapid prototyping facility allowed realistic decisions to be 
made about the viability of the ATMS concept. 

TEST .PLAN is the result of the decision to expand the por- 
tion of ATMS that provided the FTE with a planning tool. 
This system, while the development was under government 
auspices, was called the “flight test engineer’s worksta- 
tion” P] and TEST-PLAN when extended and commer- 
cialized by G & C Systems Inc. (G & C Systems Inc., San 
Juan Capistrano, CA). 
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Fig. 7 Flight system configuration. 


6 TESTJLAN 


TEST-PLAN is a computer program designed to run on 
standard graphics workstations (under either the UNIX® 
or VMS operating systems) as an aid to FTEs in planning 
and executing flight-test programs. TEST-PLAN allows 


data while satisfying planning requirements on a flight-by- 
flight basis using aircraft and flight specific information 


® UNIX is a registered trodemork of AT & T Bell Uborouxies, Whip- 
p*ny, New Jersey. 


about instrumentation, telemetry, range, center-of-gravity, 
airborne and ground support, aerodynamic configuration, 
system configuration, and payload. 

6.1 TEST-PLAN Components 

The primary components of TEST-PLAN (shown in 
Fig. 8) include: 

1. A planning facility with over 1000 flight-test plan- 
ning procedures. 
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Fig. 8 TEST-PLAN system architecture. 

2. an extensive graphical user interface (GUI), flight-test maneuvers and contingency maneuvers orga- 

nized by flights for each individual aircraft in a flight-test 

3. an interface with a relational database management ^ 0 ^^ The planning matrix is displayed in an easy to 

system (RDBMS), format (Fig. 9). 

4. an aircraft performance simulation facility. The automated planning procedures allow the FTE to 

5. a flight card generation facility, and plan flight-test programs by defining maneuvers and fill- 

ing out the planning matrix. The basic philosophy imple- 

6. expert system based planning aids. mented in the TEST-PLAN planning facility (Fig. 10) fea- 

tures the creation of a centralized database of test points 
In the following sections, we will discuss these compo- in an RDBMS which must be satisfied in the flight-test 
nents of TEST.PLAN. program; the creation of multiple flight-test plans con- 

sisting of test points assigned to flight-test maneuvers, 
flight-test maneuvers assigned to flights, and flights as- 
6.1.1 Planning Facility signed to blocks of flights for specific flight-test aircraft; 

and continuous, automated constraint checking between 
The heart of TEST-PLAN [11] is a planning facility test points, maneuvers, and flights for test point require- 

consisting of a planning matrix and over 1000 plan- ments and flight assignments on a flight-by-flight basis, 

ning procedures. The planning matrix consists of 


















29-10 




•10044 


Fig. 9 Planning matrix display. 


6.1.2 Graphical User Interface 

TEST-PLAN uses graphics extensively. One of its 
most visible features is a highly developed GUI using 
X-windows. This interface consists of windows, pan- 
els, canvasses, action buttons, and menus. Maximum 
use of is made of mouse initiated operations. The inter- 
face permits the FTE to execute procedures in any order 
desired— the FTE is not limited to the serial, predefined or- 
der of events typically found in a menu-driven application. 


Using the TEST-PLAN GUI, the flight-test engineer can 
perform many tasks which would normally require exten- 
sive paper and pencil work. These automated tasks in- 
clude laying out planning matrices, planning blocks of 
flights, defining test points, defining flight-test maneuvers, 
assigning test pants to flight-test maneuvers, sequencing 
flight-test maneuvers to minimize fuel and time required, 
writing flight cards, and satisfying test point constraints. 
These test points constraints may be based on instrumen- 
tation, aircraft configuration, range (operating area) re- 
quirements, telemetry requirements, system configuration, 
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Fig. 10 The TEST-PLAN planning philosophy. 


air and ground support requirements, payload, weight and 
balance requirements, flight limits, trim point conditions, 
or test point prerequisites. 

6.1.3 Relational Database Management System 

TEST-PLAN is functionally integrated with the Oracle Re- 
lational Database Management System (RDBMS). The 
RDBMS contains a database of test points for a flight- 
test program. TEST-PLAN incorporates many procedures 
which greatly simplify database queries, record additions, 
modifications and deletions. No special knowledge of the 
database query language is required because TEST-PLAN 
provides the user a set of menus, action buttons, and data 
entry fields as part of the GUI. 

6.1.4 Aircraft Performance Simulation 

TEST-PLAN contains a 3 dof generic aircraft performance 
simulation. This simulation requires the user to define an 
aerodynamic model (of lift and drag coefficients as func- 
tions of Mach number and angle of attack) and a propul- 
sion system model (of thrust and fuel flow as functions of 
altitude and power lever angle). 

Using the simulation and this simple definition of the vehi- 
cle, TEST-PLAN can compute the trajectory and fuel used 
in any of 52 preprogrammed maneuvers such as climbs, de- 
scents, level accelerations and decelerations, cruise, turns, 
and dynamic maneuvers. The flight-test engineer also has 
the capability of building new maneuvers by stringing to- 
gether combinations of individual maneuvers. 


6.1.5 Flight Card Generation Facility 

TEST .PLAN provides a flight card generation facility 
which uses default entries from the flight card database to 
generate a set of flight cards for a specific flight. The nom- 
inal flight card is shown in Figure 11. However, the format 
can be customized to any desired during the customization 
portion of an installation of TEST-PLAN. 

6.1.6 Expert System Based Planning Aids 

TEST-PLAN contains two expert system planning aids; 
a flight planner and a block planner. The block planner 
assigns maneuvers (which contain test points) to flights, 
a ttempting to minimize the number of flights required to 
execute the maneuvers within the block while satisfying 
constraints on instrumentation, configuration, flight lim- 
its, flight conditions, prerequisite test points, and range re- 
quirements. The flight planner reorders maneuvers within 
individual flights attempting to satisfy constraints while 
minimizing fuel and range time used; the flight planner 
uses fuel and time data obtained from trajectories gener- 
ated in the performance simulation. An explanation facil- 
ity is provided. 

7 CONCLUDING REMARKS 

This report describes the automated flight-test manage- 
ment system and an automated flight test planning sys- 
tem called TEST-PLAN. The evolution of TEST-PLAN 
from automated flight-test management system is detailed 
to illustrate the use of rapid prototyping to define system 
requirements. 
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Fig. 11 Nominal flight card format. 
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